It is known that an image of a scene viewed from a new viewpoint can be generated using a rendering technique based on a depth image (depth image based rendering, DIBR). In other words, it is possible to generate a new viewpoint image of a scene viewed from a new viewpoint, using an input image of a scene viewed from a predetermined viewpoint and a depth image representing the distance between objects in the scene. This new viewpoint image can be generated based on camera geometry using information on the camera position of an original viewpoint and the camera position of a new viewpoint. An original viewpoint image (input image) and the generated new viewpoint image can be used, for example, for 3D display as a left eye image and a right eye image. Likewise, a group of multi-view images can be generated. The group of the multi-view images can be used for multi-viewpoint 3D display.
Therefore, by using the DIBR technique, a 3D image for 3D display can be generated using an input image that is a one-viewpoint image and its corresponding depth map.
However, when a new viewpoint image is generated using an input image, an unknown region (hereinafter referred to also as a hole region) occurs in a region corresponding to an occluded region in the input image (original viewpoint image). Here, the unknown region includes no information or is empty. This is because the occluded region in the original viewpoint image is visualized (disoccluded) in the new viewpoint image. Therefore, in generating a new viewpoint image, it is necessary to fill a hole region using a hole-filling method or a disocclusion method.
Here, Patent Literature 1 suggests an in-painting method based on depth, and Patent Literature 2 suggests a hole-filling method based on the direction of image features.